Image display apparatus for displaying composite images

ABSTRACT

Image display apparatus ( 2 ) for displaying composite images, which image display apparatus ( 2 ) comprises a primary image source ( 4 ), at least one secondary image source ( 6 ), first distortion, means ( 12 ) for distorting the image geometry of the secondary image source ( 6 ) such that the image geometry of the secondary image source ( 6 ) matches that of the primary image source ( 4 ), combining means ( 18 ) for combining images from the primary image source ( 4 ) with the distorted images from the secondary image source ( 6 ), second distortion means ( 20 ) for distorting the composite images, and display means ( 22 ) for displaying the composite images.

This invention relates to image display apparatus and, more especially,this invention relates to image display apparatus for displayingcomposite images. The image display apparatus is for displayingcomposite images from more than one image source. The composite imagesappear geometrically correct to an observer when displayed on a displaysurface.

Image display apparatus is well known that is able to displayoverlapping or composite images. Computer apparatus and hardware devicesare known that utilise overlap software or firmware to produce compositeimages with overlap, capable of being displayed on single channel ormulti-channel display apparatus. The composite images can be displayedon single channel or multi-channel cathode ray tube monitor displays,liquid crystal device monitor displays, plasma displays or projecteddisplays. The projected displays may be front or rear projecteddisplays.

There are many ways of displaying composite images, and each means ofdisplay needs the images to be in the correct geometric configurationfor the chosen display means. In many instances, for example when imagesare projected onto a cylindrical screen, the composite image needs to bedistorted before being projected in order to ensure that the imageappears geometrically correct to the observer. However, when images fromdifferent image sources, for example a flat plane image source and acomputer generated out-of-the-window image source, are combined to makea composite image, then the application of distortion to the compositeimage will provide correction to the image from one image source butwill in fact introduce distortion into the image from the second imagesource. Part or parts of the display will be geometrically correct tothe observer, but the remainder of the display will appear incorrect assome distortion will have been introduced. This is particularlynoticeable in multi-channel display systems where the images extendacross joins or blend regions of the display.

It is an aim of the present invention to obviate or reduce the abovementioned problems.

Accordingly, in one non-limiting embodiment of the present inventionthere is provided image display apparatus for displaying compositeimages, which image display apparatus comprises a primary image source,at least one secondary image source, first distortion means fordistorting the image geometry of the secondary image source such thatthe image geometry of the secondary image source matches that of theprimary image source, combining means for combining images from theprimary image source with the distorted images from the secondary imagesource, second distortion means for distorting the composite images, anddisplay means for displaying the composite images.

The image display apparatus of the invention is able to displaycomposite images from various image sources such that all parts of thedisplay appear geometrically correct to an observer. The image displayapparatus may be used in multi-media presentations, virtual realitydisplays, simulation, video or computer gaming, planetaria, and variousother suitable and appropriate types of display means.

The display means may be a single-channel display means. Alternatively,the display means may be a multi-channel display means. Themulti-channel display means may be a multi-channel display means withadjacent channels having edges that are butted. Alternatively themulti-channel display means may be a multi-channel display means inwhich adjacent channels are blended.

The display means may have a flat display surface, a curved displaysurface, a cylindrical display surface, or a spherical display surface.Alternatively, the display means may be of a complex geometric shape.

The primary image source will normally have images in a specificgeometric configuration. This configuration varies with the type ofimage source. The primary image source is preferably a computergenerated image source. The primary image source may be a computer imagegenerator providing images with an out-of-the-window geometricconfiguration, as used in games or simulation. The primary image sourcemay alternatively be a computer providing images with a flat planegeometric configuration such as a desktop personal computer. The primaryimage source may alternatively be photographic still or moving imageshaving a specific geometric configuration due to lens distortion orspecialist lenses such for example as a fisheye lens. The primary imagesource may be of any other suitable and appropriate type.

The secondary image source may be any one of the above mentioned sourcesfor the primary image source. Typically the secondary image source has adifferent geometrical configuration to the primary image source. Boththe primary image source and the secondary image source may be, forexample, computer image generators, but both image sources wouldnormally have different geometric configurations, or alternatively theprimary and secondary image sources may be different types of imagesource.

The primary image source may have a geometric configuration which isspecified by the manufacturer or which may be determined by calculation.The secondary image source may also have a geometric configuration whichis specified by the manufacturer, or which may be determined bycalculation. Having determined the geometry of the primary and secondaryimage sources, the distortion means is then applied to the images fromthe secondary image source so that the geometric configuration of thedistorted secondary image source now matches the geometric configurationof the images from the primary image source. The distortion may becarried out, for example, electronically using image warping software orhardware. Other image warping techniques may be used.

The image display apparatus of the present invention may be one in whichthere are at least two of the secondary image sources, and in which eachsecondary image source undergoes the same degree of distortion prior tobeing combined with the primary image from the primary image source.Alternatively, the apparatus may be one in which there are at least twoof the secondary image sources, and in which each secondary image sourceundergoes different distortion prior to being combined with the primaryimage from the primary image source.

As indicated above, images from the primary image source and thedistorted images from the secondary image source are combined in thecombining means. The combining means may be computer apparatus orhardware utilising overlap software or firmware in order to producecomposite images with overlap capable of being displayed on singlechannel or multi-channel display apparatus.

The geometric configuration of the composite image is thus determined.All images in the composite image have matched geometries. The geometricconfiguration of the composite image matches that of the primary imagesource whose geometric configuration has previously been determined.Distortion correction is applied to the composite image. The distortionmay be carried out, for example, electronically using image warpingsoftware or hardware. The distortion applied to the composite image maybe calculated or determined such that when the composite image isdisplayed on the display means, then all parts of the composite imageappear geometrically correct to the observer. Distortion applied to thecomposite image will normally be dependent on the display means.

The display means for displaying the composite images may besingle-channel or multi-channel display means such for example ascathode ray tube displays, liquid crystal device flat panels, plasma, orfront or rear projected displays. The combining means may incorporateknown means for edge matching adjacent channels in a multi-channeldisplay, or merging the display across blend regions in a multi-channeldisplay.

Applying distortion correction to the secondary image source such thatit matches that of the primary image source before combining the images,enables the composite image to be matched to the display means. It alsoenables the composite image to be displayed such that the compositeimage is geometrically correct to the observer. If the primary image andthe secondary image are combined in the combining means, the respectivegeometries not having been matched, when the combined image is distortedand displayed, the geometries will not match and therefore part or partsof the composite image will not appear geometrically correct to theobserver.

As indicated above, the apparatus of the present invention may be usedwith more than one secondary image source. Each of the secondary imagesources may in turn be distorted to match the geometric configuration ofthe primary image source. Combining means may be used to combine theprimary image source with the distorted secondary image sources,distortion then being applied to the composite image such that thedisplayed composite image appears geometrically correct to the observer.All the composite parts, that is images from the primary image sourceand each of the secondary image sources, of the combined image thereforeappear correct to the observer.

Embodiments of the invention will now be described solely by way ofexample and with reference to the accompanying drawings in which:

FIG. 1 shows schematically image display apparatus of the presentinvention for displaying composite images;

FIG. 2 shows the geometry of a flat image;

FIG. 3 shows the geometry of an image with keystone distortion;

FIG. 4 shows the geometry of an image with barrel distortion;

FIG. 5 shows the geometry of an image with pin cushion distortion;

FIG. 6 shows figuratively distortion correction to a pin cushion image;

FIG. 7 shows figuratively distortion applied to a flat Image;

FIG. 8 shows a known three-channel display with blended images;

FIG. 9 shows a known three-channel display with a blended combinedimage;

FIG. 10 shows distortion schematically in the present invention;

FIG. 11 shows a three-channel display with blended images; and

FIG. 12 shows a three-channel display with a blended combined image ofthe invention.

Referring to the drawings, FIG. 1 shows schematically an embodiment ofimage display apparatus 2 of the present invention and for displayingcomposite images. A primary image source 4 inputs directly into imagecombining means 18. A secondary image source 6 inputs into first imagedistortion means 12. The first image distortion means 12 inputs into theimage combining means 18.

The first image distortion means 12 applies distortion to images fromthe secondary image source 6 such that the geometric configuration ofthe images matches that of the primary image source.

One or more secondary image sources may be used. FIG. 1 shows anadditional secondary image source 8 that undergoes distortion by firstdistortion means 14. FIG. 1 also shows a further additional secondaryimage source 10 that undergoes distortion by first distortion means 16.The geometrical configuration of all the image sources is common at thestage where they are input into the image combining means 18.

The composite image from the image combining means 18 is output tosecond distortion means 20. The second distortion means 20 operates suchthat the composite image is distorted such that when the composite imageis displayed on display means 22, then the image appears geometricallycorrect to the observer. The composite output image may be output on onechannel in the form of the display means 22. Alternatively, thecomposite image may be output to display means 22, 24, 26 for athree-channel display. The display means may be a single channel or amulti-channel display.

FIG. 2 shows the two dimensional geometry of an image formed in a flatimage space.

FIG. 3 shows the two dimensional geometry of an image formed in a flatimage space with a keystone distortion.

FIG. 4 shows the two dimensional geometry of an image that has a barreldistortion. Barrel distortion may be introduced into images such asvideos due to the effects of lenses that are used.

FIG. 5 shows the two dimensional geometry of a computer generated imageof the world. This is representative of the three-dimensional world andimages of this type need to undergo distortion before being displayed byprojection on a flat screen.

FIG. 6 shows the distortion that is applied to the geometric imageconfiguration shown in FIG. 5. This distortion is applied such that theimage appears correct to an observer when viewed on a screen. Theoriginal image 28 is distorted via a function represented by 30. Theresultant image geometry is indicated by 32. If a complementarydistortion 30 is used, then the resultant image will appear correct tothe observer. The distortion applied by distortion means 30 will varywith respect to the initial distortion of the image and with the finaldisplay means chosen.

FIG. 7 shows that an image 34 with a regular geometric configuration canhave a distortion 36 applied to it, in order to produce a final image 38that is distorted. This final image 38 will not appear to be correct tothe observer and it will appear to be warped.

FIG. 8 shows known three-channel display means 40 with blend regions 48and channels indicated by 42, 44 and 46. The geometric configuration 50of the left channel is shown. This configuration has been produced inaccordance with the distortion procedure mentioned above in connectionwith FIG. 7. The image in each channel has some distortion. Thedistortion in each channel need not necessarily be the same.

FIG. 9 shows known three-channel display means 52 with channelsindicated by 54, 56 and 58 for displaying composite images. Thegeometric configuration 62 of the left channel is shown. Thisconfiguration has been produced in accordance with the above mentioneddistortion procedure described for FIG. 7. Part of the composite image60 is shown. The distortion introduced is particularly noticeable acrossthe blend region. In composite images, this part of the image 60 may be,for example, a desk top personal computer, and would appear distorted tothe observer. The background or surrounding images may appear correctsince the distortion applied to the composite image would correct thispart of the composite image.

FIG. 10 shows image display apparatus of the present invention fordisplaying combined images. The image display apparatus has a primaryimage source 64, with images in an associated geometrical configurationas shown. The image display apparatus has a secondary image source 66,with images in an associated geometrical configuration as shown. Theimage display apparatus has distortion means 68 for distorting theimages from the secondary image source. The image display apparatus alsohas combining means 70 for combining the images from the primary imagesource with the distorted images from the secondary image source.Distortion means 72 is employed for distorting the composite images.Display means 74 is employed for displaying the composite image. It canbe seen that the images have the same geometric configuration when inputinto the combing means 70. Consequently, any distortion applied to thecomposite image has the same effect on all parts of the composite image,and the composite image is displayed with all parts appearing to becorrect to the observer.

FIG. 11 shows three-channel display means 80 for use in the imagedisplay apparatus of the present invention. The three-channel displaymeans 80 has blend regions 90, and channels indicated by 82, 84 and 86.The geometric configuration 88 of the left channel is shown. Thisgeometric configuration has been produced in accordance with thedistortion procedure described above with reference to FIG. 10.

FIG. 12 shows three-channel display means 92 for use in displayapparatus of the present invention. The three channels are indicated by94, 96 and 98. The geometric configuration of the left hand channel isindicated by 100. Part of the composite image is indicated by 102. Thispart of the composite image has no distortion, and it appearsgeometrically correct to the observer, especially across the blendregion.

It is to be appreciated that the embodiments of the invention describedabove with reference to the accompanying drawings have been given by wayof example only and that modifications may be effected.

1. Image display apparatus for displaying composite images, which imagedisplay apparatus comprises a primary image source, at least onesecondary image source, first distortion means for distorting the imagegeometry of the secondary image source such that the image geometry ofthe secondary image source matches that of the primary image source,combining means for combining images from the primary image source withthe distorted images from the secondary image source, second distortionmeans for distorting the composite images, and display means fordisplaying the composite images.
 2. Image display apparatus according toclaim 1 in which the display means is a single-channel display means. 3.Image display apparatus according to claim 1 in which the display meansis a multi-channel display means.
 4. Image display apparatus accordingto claim 3 in which the multi-channel display means is a multi-channeldisplay means with adjacent channels having edges that are butted. 5.Image display apparatus according to claim 3 in which the multi-channeldisplay means is a multi-channel display means in which adjacentchannels are blended.
 6. Image display apparatus according to claim 1 inwhich the display means has a flat display surface.
 7. Image displayapparatus according to claim 1 in which the display means has a curveddisplay surface.
 8. Image display apparatus according to claim 1 inwhich the display means has a cylindrical display surface.
 9. Imagedisplay apparatus according to claim 1 in which the display means has aspherical display surface.
 10. Image display apparatus according toclaim 1 in which the primary image source is a computer generated imagesource.
 11. Image display apparatus according to claim 1 in whichsecondary image source is of a different geographical configuration tothe primary image source.
 12. Image display apparatus according to claim1 in which the secondary image source is a computer generated imagesource.
 13. Image display apparatus according to claim 1 in which thereare at least two of the secondary image sources, and in which eachsecondary image source undergoes the same degree of distortion prior tobe combined with the primary image from the primary image source. 14.Image display apparatus according to claim 1 in which there are at leasttwo of the secondary image sources, and in which each secondary imagesource undergoes different distortion prior to be combined with theprimary image from the primary image source.
 15. (canceled)